test: functional test config and soak test#135
Merged
Conversation
corpo-iwillspeak
requested review from
anna-singleton-resolver,
Copilot and
tcarroll-kroll
corpo-iwillspeak
force-pushed
the
test/functional-test-config-and-soak
branch
from
fc70243 to
22cdcb4
Compare
…oak test Env vars (ATHENA_HOST, OAUTH_AUTH_URL, OAUTH_AUDIENCE) now fall back to their SDK defaults when unset rather than overriding with stale hard-coded values. Adds proactive_refresh_threshold to CredentialHelper and a new classify_single soak test that validates hash-check classification success rate over 1000 iterations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
corpo-iwillspeak
force-pushed
the
test/functional-test-config-and-soak
branch
from
22cdcb4 to
1067cee
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the functional-test configuration to rely on SDK defaults when certain env vars are unset, and adds a new long-running soak test for classify_single to track KnownCSAM-* hash-check classification presence over many requests.
Changes:
- Functional test config: stop hardcoding defaults for
ATHENA_HOST,OAUTH_AUTH_URL, andOAUTH_AUDIENCEand let SDK defaults apply when unset. - Functional test auth: adjust
CredentialHelperconstruction (incl. proactive refresh threshold setting). - Add a new soak test that runs 1000
classify_singlecalls and asserts a minimum hash-check classification success rate.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/functional/test_classify_single_soak.py | Adds a new soak test for repeated classify_single calls and success-rate assertion for KnownCSAM-* classifications. |
| tests/functional/conftest.py | Updates functional-test env var handling to avoid overriding SDK defaults; adjusts how CredentialHelper and AthenaOptions are constructed. |
Comments suppressed due to low confidence (1)
tests/functional/test_classify_single_soak.py:67
- The assertion message hard-codes "below 95%" even though the threshold is defined by
MIN_HASH_CHECK_SUCCESS_RATE. If the constant changes, the message becomes misleading.
assert success_rate >= MIN_HASH_CHECK_SUCCESS_RATE, (
f"Hash check success rate {success_rate:.1%} is below 95%. "
f"{len(failures)} failures:\n" + "\n".join(failures[:20])
)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tcarroll-kroll
approved these changes
Jun 26, 2026
anna-singleton-resolver
approved these changes
Jun 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ATHENA_HOST,OAUTH_AUTH_URL, andOAUTH_AUDIENCEin functional test config — these now fall back to SDK defaults when unset, rather than overriding with stale hard-coded valuesproactive_refresh_threshold=0.25toCredentialHelperconstruction so tokens refresh before expiry during long test runstest_classify_single_soak.pythat sends 1000classify_singlerequests and asserts a ≥95% KnownCSAM- hash-check classification success rateTest plan
pytest -m soak) to validate the success-rate assertion🤖 Generated with Claude Code